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AN INTERACTIVE. COMPUTER-AIDED 
SPEECH EDUCATION METHOD AND APPARATUS 

BACKGROUND 

The present invention relates generally to educational methods at the classroom and 
5 individual level, as well as systems for implementing such methods. More particularly, the 
present invention relates to an educational method which combines audio-visual and computer 
technology to shift the teacher's role away from mass pupil instruction and paperwork and 
back to the more traditional role of helping each individual student with his or her unique 
educational difficulties. Of particular significance in this application are methods and 

10 apparatus directed to teach speech/language skills to students. 

Classroom teaching methods have changed very little since the early days of our 
country. The bedrock of education remains the teacher or professor lecturing to groups of 
students, conmionly in conjunction with a text. Homework is assigned to reinforce the lessons 
learned in class and tests are administered to measure students' aptitude and retention of the 

15 material. At the early levels of education, the setting for this instruction has and continues to 
be the schoolroom, public or private, where a teacher is responsible for a group of, for 
example, fifteen to forty students. 

The conventional classroom method of teaching a subject typically involves a lesson 
plan which includes a plurality of lectures each having an associated written homework 

20 assignment for the student to reinforce the lecture. The teacher collects and evaluates the 

homework and occasionally gives tests to measure the students' retention of the material. The 
teacher is also required to perform a multitude of other subsidiary tasks, such as report card 
generation and parent-teacher conferences, to keep the community satisfied with both the 
students* and the teacher's performance. Thus under the conventional classroom scenario, 

25 teachers must split their time to assume the roles of administrator and public relation officer, 
as well their supposedly primary role of teacher. 

Educators have long recognized that while the classroom scenario provides economies 
of scale, one drawback is that students of different ages and aptitudes learn at differing rates 
and with the need for varying amounts of individual attention. This recognition led to the 

30 availability of private tutoring, for those who could afford such, provides more flexibility to 
meet an individual student's needs. On the other hand, private tutoring has also been 
criticized since it does not provide the student with the opportunity to develop social skills 
which is inherent in the classroom setting. 
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Although this traditional method of education has changed little over the years, the 
amount of information which students need to assimilate to be considered well educated 
continues to increase geometrically. This fact is graphically illustrated by the technological 
wonders which surround us such as microcomputers, DNA mapping, videophones, the 

5 Internet, etc. The increase in information puts a correspondingly greater burden on teachers 
to try to cram more learnmg into the same amount of time, which leads to an associated 
increase in the amount of paperwork to be handled. Moreover, as fiscal resources grow more 
and more scarce, teachers have been required to handle even larger groups of students 
resulting in even less individiial attention. 

10 Although these problems are rapidly becoming more acute, they have been 

acknowledged by educators for many years. Many countries have reacted by adding more 
days to the school year or sending children to school at an earlier age so that students have 
more tune to acquire the additional knowledge necessary. This solution, however, will only 
provide short term relief since there is only a finite amount of additional learning time which 

15 can be reasonably added to students' schedules, while our knowledge base will always 

continue to expand. Additionally, increasing the school year fiirther aggravates the fiscal 
problem of education by requiring even greater economic resources and putting greater 
pressure on teachers. 

Educators have also experimented with alternate teaching methods, such as self-study 
20 programs and videotaped lectures, to both more efficiently use teachers' time and to provide 
an element of individuality to a student's smdy program. Although these types of methods are 
somewhat successftil at higher levels of education, they are generally ineffective at lower 
levels where younger smdents lack the discipline to teach themselves. 

Another approach has been outlined in the afore-incorporated by reference patents to 
25 John Lee et al. Therein, interactive educational methods and systems are described which 
provide a flexible alternative to the conventional classroom educational method. These 
educational methods integrate technology and learning in a manner which is specifically 
designed to make learning more effective for both students and teacher. For example, the 
computer-aided techniques describe therein relieve the teacher of many of the non-teaching 
30 burdens discussed above, while also giving the smdent the opportunity to learn at his or her 
own pace. These systems also provide an interactive, audio-visual system which will 
encourage each smdent to learn at his or her own individual pace and using teaching 
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techniques that are adapted to promote learning according to each individual's strengths and 
weaknesses. Moreover, each student is provided with instructional reinforcement in the form 
of questions and his or her performance will be monitored by the system. If a smdent has 
difficulty with a particular area, the system will alert the teacher who can then provide 
additional help. 

Another area of teaching in which it would be desirable to provide creative, computer- 
aided solutions is speech/language education. This type of education focuses on teaching 
students how to correctly pronounce various words and/or phrases. Recent technological 
advances in the area of voice signal processing and speech recognition have provided the 
building blocks upon which computer-aided educational techniques can advance into the realm 
of speech/language education. Actually teaching students how to pronounce words, however, 
is more challenging than simply training a computer to recognize speech that is input thereto. 
The present invention addresses this challenge and provides methods and systems for 
interactive, computer-aided speech education. 

SUMMARY 

In accordance with the present invention, an educational speech teaching method using 
an interactive audio-visual computer system includes outputting, by a computer system, at 
least one example of a speech object; prompting, by the computer system, a student to repeat 
the speech object; storing the repeated speech object in the computer system; evaluating the 
stored, repeated speech object using a speech evaluation engine to generate grades associated 
with the stored, repeated speech object on a phoneme level; and determining, by the computer 
system, whether to provide another example of the speech object or to output a positive 
reinforcer based on the grades generated by the speech evaluation engine. 

BRIEF DESCRIPTION OF THE DRAWINGS 

These and other advantages and objects of the invention will become more apparent 
from the following detailed description of the preferred embodiments when read in conjunction 
with the attached Figures in which: 

Figure 1 illustrates a block diagram of the elements of a system for implementing the 
educational method according to an embodiment of the present invention; 
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Figure 2 sets forth a flow chart which indicates an overall sequence of events according 
to a preferred embodiment of an educational method according to the present invention; 

Figures 3 and 4 illustrate flowcharts describing subprocesses of the sequence described 
in Figure 2; 

5 Figure 5 is a flow chart illustrating another exemplary embodiment of the present 

invention; 

Figures 6-16 depict exemplary interface screens of a teacher's workstation according to 
exemplary embodiments of the present invention; 

Figure 17 is a flowchart describing an exemplary speech education method according to 
10 an embodiment of the present invention; and 

Figure 18 is a block diagram depicting the general hardware and software elements 
associated with implementing the method of Figure 17. 



DETAILED DESCRIPTION 

In the following description, for purposes of explanation and not limitation, specific 
15 details are set forth, such as particular circuits, circuit componems, techniques, etc. in order to 

provide a thorough understanding of the present invention. However, it will be apparent to 

one skilled in the art that the present invention may be practiced in other embodiments that. 

depart from these specific details. In other instances, detailed descriptions of well-known . 

methods, devices, and circuits are omitted so as not to obscure the description of the present 
20 invention. 

A system for implementing the educational method according to an exemplary 
embodiment of the present invemion will be described with reference to Figure 1 . In this 
exemplary embodiment it is envisioned that each student will be provided with a workstation 
10 that may comprise all of the devices indicated in block form within the dotted lines of 
25 Figure 1 linked together via system bus 12. The workstation 10 can, for example, be 
microcomputer and the CPU 14 could be that which is part of the microcomputer. For 
example, an IBM PC having an Intel 80386. 80486 or Pemium* microprocessor could provide 
the platform for the workstation. Alternately, more or less powerful computer systems could 
be used. 
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The workstation 10 can be integrated within or around an ergonomic environment. 
Each station can include an ergonomically designed chair (not shown) so that the student can 
remain comfortably seated for extended periods of time. The display 16, which can be a 
conventional VGA monitor attached via a serial port to the system bus 12, can be mounted 
5 within a student's desk under glass at about a 45 degree angle. By providing workstations 
having the display 16 mounted under glass, the classroom as a whole is uncluttered and the 
teacher is able to readily observe both students and their respective workstation displays. 
Alternately, the display can simply rest on the desktop. In some preferred embodiments of the 
invention, the display will be touch sensitive so that a student can enter data and respond to 

10 questions from the computer via the screen. 

The keyboard 18, floppy drives 20, and hard drive 22 are all conventional components 
attached via suitable interfaces to the CPU 14. The auxiliary input device 24 can be any type 
of pointing device such as a mouse, digitizing tablet or light pen. The CD-ROM device 26 
and laser disk 28, for providing text data and video data, respectively, are also of a 

15 conventional design and can be either built-in to the workstation or provided as sumd-alone 

models. Full screen video presentations, or partial screen windows or video presentations, can 
be stored on CD or laser disk using, for example, an IMPEG compression technique. 

Each of the TV input module 30, audio interface 32, and stereo input module 34 can be 
off-the-shelf boards which are designed to plug into the expansion slots of a microcomputer in 

20 a known manner. The TV input module 30 is a board whiwj allows a computer to receive 
television signals via either antenna or cable and convert those signals into image signals 
which can be displayed on all or part of the display 16. The audio interface 32 is an 
input/output board which allows the computer to send sounds, including voice and music, to a 
student via headphones (not shown) which plug into the board and receive such sounds via a 

25 microphone (not shown). Alternately, the audio interface could simply be a speaker which 
would be connected to a sound board and/or a voice synthesizer as is well known in the art. 
The stereo input module 34 is a board which allows the microcomputer to receive radio 
transmissions or signals, from a tape player, CD player or any other type of audio device, 
which a student could listen to as background music. Printer 36 can be any type of 

30 microcomputer compatible printer, including dot matrix, inkjet or laser printer, which 
conraiunicates with the CPU in a known manner. 
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AU of the student workstations are in constant communication with a teacher's 
workstation 40 via a LAN interface 42 and local area network (LAN) 44. As described in 
more detail below, this real-time communication between smdent workstation and teacher 
workstation allows the teacher to be informed of the students' progress and activities as well as 

5 allowing the teacher to tailor instructional programs for each student. 

The foregoing description will be understood by one skilled in the art as merely an 
example of one system which could be used to implement the various embodiments of the , 
educational method of the present invention described in detail below and that various other 
configurations and devices could also be used. Reference to the conventionality of the various 

10 I/O devices described above is made merely to emphasize that one skilled in the art will 

recognize that the system described in Figure 1 can be implemented using known interfaces 
and software techniques. 

Initially, one advantageous feature of the educational method according to the present 
invention is that courseware can be uniquely developed for each class, school, school system 

15 or any other segment of a smdent population so as to provide a curriculum desired by the 
educators using the system and method of the present invention. The term "courseware" as 
used throughout this specification is meant to comprise the combination of control programs 
and data which provide each lesson to the student at a workstation and includes, for example, 
both the information stored on the hard drive, laser disk, and CD-ROM, as well as that input 

20 through the TV input module, and the programmed presentation of that data. The phrase 

"lesson segment" is used to denote a block of material presented to a student which is intended 
to be viewed and interacted with in one sitting, typically 15-40 minutes. A lesson segment can 
also be a test which evaluates the student's retention of one or more lesson segments or a 
whole semester's worth of material. 

25 As a simplified example, history courseware dealing with the American Revolution 

might comprise a plurality of lesson segments. One lesson segment might include, for 
example, a narrative story of George Washington's role in the revolution. The text of the 
story, read from the CD-ROM, might occupy one half of the display while illustrations, from 
the laser disk, occupy the other half of the screen. After each segment of the lesson, the 

30 program could test the student's comprehension by asking the student to answer a series of 
questions via the keyboard which relate to the previous segment of the lesson. 
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Thus the teacher or school system can select material including, for example, text, 
illustrations, length of lesson and questions to be answered, to comprise the courseware for 
any subject. This courseware development step in the educational method according to the 
present invention advantageously provides teachers and parents with the opportunity to create 
5 and update the educational material for presentation to the students. 

One skilled in the teaching art will appreciate that many other teachmg tools could be 
integrated into the courseware, in addition to those mentioned above, to present various topics. 
For example, some topics, such as biology, are particularly conducive to teaching via film 
presentation. This material can be input via the TV input module, or presented as computer 

10 video decompressed from a CD, and displayed while an accompanying voice description is 
listened to by the student on the headphone set. 

Next, Figure 2 illustrates an exemplary preferred embodiment of an overview of an 
educational method according to the present invention wherein the student is using the 
workstation and the system has been loaded with the developed courseware. In block 100, the 

15 teacher initializes the system and then the smdents enter their homework assignments from the 
previous day into the system at block 1 10. The homework assignment can be entered 
manually by the student typing the answers on the keyboard. Alternately, an optical scanning 
device (not shown) can be provided which is used to scan the homework paper and digitize the 
answers. In either case, the homework answers are compared to the expected answers by the 

20 CPU of each workstation and the results transmitted via the LAM to the teacher's workstation 
as indicated by block 120. The homework answers are stored in database files corresponding 
to each smdent name or ID number in the hard drive at the teacher's station for future use. 

The teacher and/or a system program can then use the results of the homework 
assignments in conjunction with each student's progress, which has been stored in another 

25 database file at the end of the previous day, to assign lesson segments to each student as the 
process flow continues to block 130. Alternatively, lesson assignments can be determined 
solely on the basis of each student's progress and homework can be used either as a measure 
of retention or as a pretest. This assignment process allows the teacher and/or a system 
program to determine how much and what type of material each student can access for a given 

30 period of time, and provides the first of several opportunities for the teacher and/or system 
program to tailor each student's individual learning program. 
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Within the system, this assignment process is controlled by the CPU of the teacher's 
station which downloads the control programs corresponding to the lesson segments selected 
by the teacher and/or a system program from the hard drive or other storage device of the 
teacher's workstation to the selected smdent's station through the LAN. Alternately, a single 

5 control program could be downloaded to, or stored on, the students' workstations which will 
allow selection of lesson segments stored on the hard drive of each student's workstation. 
When run by the smdent these control programs access the various information storage devices 
to retrieve the audio and visual data created for each lesson segment. 

In block 140, the process flow moves to the next step of displaying the main screen at 

10 each student work station. In addition to lesson segments, this screen can also include 
selections such as games, movies or educational video programs, and creative writing or 
drawmg activities. These selections can also be chosen by the teacher, student, and/or a 
system program and can be downloaded through the LAN to each student workstation. As 
mentioned briefly above, depending on the educational level at which the system and metjiod 

15 are being implemented, the smdent can make selections in a variety of ways. Younger , 
smdents, for example, can make a selection by touching color-coded figures displayed op a 
touch-sensitive screen. More advanced students can make a selection by using the 
workstation's pointing device to click on the appropriate icon. 

In decision blocks 141, 143, 145, and 147, the process flow checks to see which 

20 selection is made from the main screen. For exemplary purposes only, the choices of . 

LESSON SEGMENT, GAME, VIDEO and END are illustrated in Figure 2, however it is to 
be understood that many other different types of selections can be provided. Based on the 
determined selection, an appropriate subroutine 142, 144, 146, or 148 is executed. The 
LESSON SEGMENT selection will now be described with reference to Figure 3 which 

25 illustrates an exemplary flowchart breaking down some of the steps of a lesson segment 
subroutine as diagrammatically indicated at block 142. 

A control program corresponding to the selected function which was previously stored 
on the hard drive or other storage device of the student's workstation during the assignment 
process is initiated in block 152. Prior to presenting a lesson, a pretest can be performed to 

30 determine the stodenfs current level of comprehension of a subject. If the student's 

comprehension is sufficiently high, then the lesson presentation may be skipped and the 
smdent can be quizzed on this topic. Otherwise, the smdent then watches and listens to the 
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audio-visual presentation created by the control program as the process flow continues in block 
153. For example, a series of mathematics problems may be presented on display 16 using 
computer-driven video of a teacher's prerecorded lesson. The control program will also 
provide software control options for the student such as adjustment of speed or volume of the 
presentation and the ability to stop and restart the presentation so that the smdent can take a 
break or ask the teacher a question as discussed below. 

At decision block 154 the control program checks to see if it has presented all of the 
material for the lesson segment. If so, the subroutine ends and the control program goes to 
block 180 in Figure 3 to execute a quizzing subprocess described below. If not, the control 
program polls various flag variables associated with the commands available to the student 
during the presentation as shown at decision block 156. If a flag has not been set, the control 
program loops back and continues displaying material at block 153 and then again checks if 
the lesson segment is over. If a flag has been set, indicating that the student wants to change 
or stop the presentation, the flow proceeds to a decision tree to implement the corresponding 
subroutine. 

A first such exemplary subroutine is shown beginning at decision block 158, wherein it 
is asked whether the flag representing the STOP command was set. If not, the flow proceeds 
to the next command checking decision block 160. If the smdent has activated the stop 
command, the process flows to decision block 159 where the student is asked if he or she 
wishes to save the current position in the lesson segment and return the main screen. If so, 
a pointer indicating a last displayed time frame in the lesson segment is stored and the process 
goes back to block 140 in Figure 2. Otherwise the presentation is temporarily frozen at block 
162. Next, the control program checks to see if the student has activated the restart command 
at decision block 164. If so, the process flow remrns to block 153 and continues to display 
material in that loop. If not, a counter is incremented at block 166 and the count is compared, 
at block 168, to a predetermined number, for example, a number equal to 5 minutes, to see if 
the teacher should be notified of the delay. If the current number is equal to the 
predetermined number, a message is sent over the LAN to notify the teacher at block 170, 
otherwise, the process flow loops back to again check if the restart command has been given. 

The SPEED and VOLUME control commands are checked for and implemented at 
blocks 160, 172, 174, and 178 in a similar manner. Since these functional subroutines are not 
themselves part of the present invention and are well known in the art, they are not described 
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in further detail herein. Moreover, one skilled in the art will appreciate that a plurality of 
other commands can be made available to the student by simply adding such subroutines to the 
decision tree and providing icons or areas of a touch-sensitive screen which will set a flag 
corresponding to that function. 
5 After a lesson segment has finished being presented, the control flow according to this 

exemplary embodiment of the present invention goes from decision block 154 to block 180 
which executes a quiz subroutine illustrated in more detail in Figure 4. At the end of each 
lesson segment, which can be designed at the courseware level to be of an optimal attention 
span length for the intended grade level of the student, typically there will be a series of 
10 questions retrieved from a database of questions associated with the particular lesson segment 
for the student to answer at block 182. The student enters his or her responses via the 
keyboard and the workstation CPU compares these responses to the correct answers stored in 
the database at blocks 184 and 186. The quiz can be time limited as discussed below. If 
desired, a grade on the smdent's responses can be generated and transmitted to the teacher's 
15 station for storage in the student's file. 

At decision block 186, if the student has correctly answered all of the questions the 
flow loops back to the main screen block 140 of Figure 2, whereat the student can access 
another lesson segment, request recreational material, or end for the day. If the smdent has 
incorrectly answered some questions, the process flow loops back to block 153 of Figure 3 
20 and the program reu-ieves and replays only material relating to those questions which were 
missed. This replayed material could be excerpted from the original presentation or it could 
be new material specifically designed to explam the correct answer to each incorrectly 
answered question. The process of replaying this material includes all of the steps discussed 
above with respect to subroutine block 142 and the subroutines of Figure 3. Alternatively, the 
25 presentation of remedial material might occur only when a smdent' s quiz score is below a 
predetermined threshold. 

After playing this remedial material, the same or different questions as those previously 
answered incorrectly will again be displayed and answered at blocks 182 and 184. The 
workstation CPU will again check the answers at decision block 186 and transmit the results to 
30 the teacher's station for storage. If all of the answers are correct, the process flow returns to 
the main screen block 140. If, however, after a predetermined number of tries the student still 
fails to grasp the material and answers some questions incorrectly, the process flows to block 
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190 and the student's workstation will send a message to the teacher's workstation indicating 
which material the student is having problems with. The teacher can then use his or her own 
methods to personally help the student to grasp the material. 

According to another exemplary embodiment of the present invention, a curriculum 
5 database provides different ways of presenting the concepts contained in lesson segments. For 
example, the same lesson segment can correspond to plural presentations stored in the 
curriculum database, each of which is based on a different teaching style, format and/or 
media. Thus, each lesson segment will have corresponding presentations stored in the 
curriculum database which emphasize, for example, voice or other audio information, 

10 pictures, animation, symbolism, analogy, repetition, and combinations thereof. Of course 
those skilled in the art will appreciate that these are only a few of the different media and 
different teaching styles which can be combined to provide different presentations for a 
particular lesson segment. 

An exemplary usage of a curriculum database according to the present invention will 

15 now be described. Suppose that a student is working on a lesson segment in Algebra that is 
intended to convey the relationship between the equation y = mx + b and a line on a graph. 
The student first works with an animated presentation that illustrates how the equation y = x 
+ 1 corresponds to a line which has slope 1 and intersects the y-axis at coordinate (0,1). For 
whatever reason, the student does not master this topic after working with this lesson segment, 

20 this lack of mastery indicated, for example, by the quizzing subroutine of Tigure 4. 

The system then branches back to block 153 of Figure 3, as earlier described, for the 
presentation of additional material. In this case, the additional material comprises all or part 
of a different presentation which corresponds to this Algebra lesson segment. For example, 
the system could provide a presentation which uses an audio and "hands on" approach, taking 

25 the student step-by-step through a process that asks the student to draw a graph on a piece of 
paper, write down the equation y = x + 1 and draw in points which come from the equation. 
This particular student, who happens to grasp things better by actually doing them, now 
understands the concepts which eluded him or her after the first presentation and subsequently 
passes the next quizzing segment so that he or she can move on to new challenges. 

30 Of course the foregoing is merely an example, and these presentations could have been 

described as being in the opposite order, the point being that different approaches to 
presenting the same material are provided by the curriculum database to prevent frustration on 

BNSDOCID: <WO 0ie2264Al J_> 



wo 01/82264 PCT/USOl/13258 

-12- 

the part of the student and accelerate the learning process. Moreover, the system can learn the 
idiosyncracies of the individual students and present lesson segments accordingly. Thus, if for 
example some students grasp material more quickly based on audio presentations, their first 
presentation of a lesson segment can be one which has been predetermined to emphasize audio 

5 stimuli. If, on the other hand, other students react better to visual presentations, their first 
presentation can be one which has been predetermined to emphasize visual stimuli. 

The system can learn the idiosyncracies of the students by conventional means, e.g., 
artificial intelligence, expert systems, simply generating statistics based on the number of 
questions answered correctly after working with each category of presentation type, etc. This 

10 information can be compiled and stored in the LAN 44 or the teacher's workstation 40 and 

used to generate a preference ranking that tailors the order of initial presentation and remedial 
presentations for each student. Alternately, or in combination with this automatic method, the 
teacher can designate this order or any part thereof for all or any number of students. The 
teacher can also temporarily override the presentation preferences of each student. If, for 

15 example, it is desired to test a new presentation package which has been added to the 

curriculum database, then the teacher could override the presentation ranking so that each 
student first accesses the new package for a particular lesson segment. 

In addition to lesson segments, blocks 144 and 146 generally indicate game and video 
subroutines, respectively, which can be selected by a student from the main screen if they have 

20 been sent to the student workstation by the teacher and/or a system program. These selections 
can be used to reward the student or merely to provide a break from the more educational 
materials. Like the lesson segment function described above, the selection of these functions 
initiates a control program which allows the student to play a game or watch a prpgram for a 
predetermined time. After the predetermined time has elapsed, the program will return the 

25 student to the main screen where only lesson segment selections will be available so that the 
student can complete the assigned lessons for the day. Further detail regarding particular 
games and video programs and their control are beyond the scope of this disclosure and can be 
readily implemented using conventional means. Again it should be noted that games and 
videos are detailed herein only as examples of other ftinciions which can be integrated into the 

30 system and method according to the present invention and that many other selections and 
ftmctions can be readily implemented by one skilled in. the art. 
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The END function which is recognized by decision block 147 and implemented at 
block 148 of Figure 2 can be selected by the student at the end of the day or as a logoff feature 
if other students need to use the workstation. Housekeeping software will update the database 
files at the teacher's station regarding the student's activity during the session being 
5 terminated. Homework for the following day corresponding to the lesson segments completed 
during the session can be can be retrieved from memory and generated by the printer. 
Additionally, progress reports or remedial notes can be generated at the printer for the student 
to take home to his or her parents. ' 
The teacher's workstation 40 can comprise a system similar to that of the student 

10 workstation 10 which may be enhanced to provide sufficient processing resources to control 
the LAN and provide monitoring capabilities of each of the student workstations tied into the 
LAN. The teacher's workstation can include a more powerful CPU, larger on-board memory 
and a larger hard drive to store, for example, all of the various databases used to store 
information received from the students' workstations. In addition to the functions discussed 

15 above, the teacher's workstation also provides the teacher with other capabilities including, 
editing the databases associated with the materials and questions of the lesson segments and 
report generation features which allow the teacher to easily generate grade reports on any or 
all of the students. An exemplary teacher's workstation will now be described with respect to 
Figures 6-16 which show exemplary interface screens. 

20 Figure 6 depicts a first menu from which a teacher may select various submenus, the 

classroom shutdown command or the quit command. The class shutdown command provides a 
way in which the teacher can signal the students' workstations that, for example, the end of a 
class period is approaching and that students should not be permitted to access additional 
lesson segments which might extend beyond the end of the classroom period. The classroom 

25 setup submenu is illustrated in Figure 7. Therein various functions including adding or 

changing a student's information, deleting a student, printing the classroom lists, and printing 
logon names are provided. An exemplary screen for adding or changing a student appears as 
Figure 8. 

The lesson semp or lesson maintenance menu is illustrated as Figure 9. Each of the 
30 first five options listed from the top of this menu have examples thereof shown in Figures 10- 
14. By clicking on the print lesson information option, the teacher has the ability to capture 
information regarding the lessons in printed form. As can be seen in Figure 10, each lesson 
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segment can be presented to the students in any order desired by the teacher. For example, 
the teacher could change the numbers indicated in the "lesson order" colunm to change the 
order of lesson presentation based upon his or her teaching experience. 

Figure 1 1 shows an exemplary chapter test location menu which allows the teacher to 
selectively determine when quizzing will be performed relative to the presentation of the 
lessons. Additionally, a teacher can exclude certain lessons from the quiz as desired. Figure 
12 shows an exemplary interface which can be used for the teacher to determine what score on 
a quiz will allow a student to proceed to the next lesson. For example, a threshold score can 
be entered in the "Grade" column which will allow a student to proceed to the next lesson. 
Each quiz can have its own score for passing individually determined by the teacher or a 
universal threshold could be provided for all quizzes. 

Figure 13 displays an interface which can be used at the supervisory station to provide 
a maximum quiz time within which period a student must finish taking a quiz. At the end of 
this maximum time, the system would then evaluate the student's current responses and ^ 
provide a grade or score. The homework editor screen of Figure 14 can be used by the 
teacher to associate various assignments with each lesson presented by the system. Thus, at 
the end of a class (as indicated by the teacher shutting down the classroom) homework can be 
generated based upon the entries and changes made by teacher on this screen. , , 

Figure 15 illustrates a gradebook menu screen which allows the teacher to easily output 
grades either as a hard copy or to the teacher's display. Although the screen shows two 
examples of ways in which grade information can be output, those skilled in the art will 
readily appreciate that the grade database can be queried in a number of different ways to 
produce different reports. One exemplary report is illustrated as Figure 16. In this screen, 
the smdents are alphabetized and their scores and progress through each lesson segment is 
both graphically and numerically portrayed so that a teacher can use this gradebook screen to 
quickly determine which smdent or students are having problems with which particular lesson 
segments, i.e., by looking at the color coded indications on the screen. 

According to another exemplary embodiment of the present invention, speech education 
can also be provided as a feamre of the aforedescribed system. As mentioned above, speech 
education according to the present invention refers to techniques which are more interactive 
and dynamic than conventional speech recognition. An example will be provided using the 
flowchart depicted in Figure 17 as well as the concepmal block diagram of Figure 18 which 
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identifies some of the hardware and software components involved in the speech education 
techniques according to the present invention. 

First, speech education according to the present invention provides the ability for the 
system to provide one or more visual and/or audible examples of phonemes, syllables, words, 
5 phrases or sentences that are the object of the lesson. This can be accomplished by, for 
example, an object word "theatrical" being displayed on display 16 and a correct 
pronunciation thereof output as synthesized speech from audio interface 32 (e.g., via 
speaker(s) 1800). As with the earlier described embodiments, the manner in which examples 
are provided by the system, e.g., a straightforward visual representation of the word, a picture 

10 reinforcement illustrating the word, a visual representation of the word made up of its 

composite phonemes (as in the dictionary, typically to the right of the word itself), an audio 
segment of a speech professional saying the word or an audio segment of a speech professional 
saying each phoneme that makes up the word individually, can be established by the teacher 
using the teacher's workstation 40 to set the parameters for the speech education class, at the 

15 student's own workstation 10, or by the system based upon the student's own preferred 

learning style as reflected by a pattern of correct/incorrect responses. For example as in the 
previous embodiments, the system can initiate the lesson using a first style of example (audible 
repetition of the object word) and, for failed attempts, can repeat the example with a 
reinforcing example which emphasizes a different stimuli (e.g., a visual or audible 

20 phonetically broken down version of the word). Moreover the number of repetitions of each 
example is also variable by the system, e.g., under the control of the teacher at the teacher's 
workstation 40. 

The example or examples can be provided to the student in conjunction with the the 
courseware's display of educational aids on the display 16 via graphical user interface (GUI) 

25 1804. In this instance the courseware is referred to as application software layer 1806 to refer 
to its interface and control ftmctionality for speech engine 1808 (described below) and GUI 
1804. The speech objects of interest can, for example, appear to be pronounced by an 
educational character displayed on display 16. Once the example or examples have been 
provided, the smdent will be prompted to repeat a speech object at step 1702. This prompt 

30 can also be visual, audible or both. Additionally, it may be useful to prompt the student to 

repeat the speech object multiple times, e.g., in order to allow the system to grade and average 
multiple responses to provide a more accurate assessment of the student's ability to properly 
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pronounce the object. The student's responses are captured by the system, e.g., by a 
microphone 1802 which is included in audio interface 32 of the student's workstation. 
Depending upon the ambient conditions, various approaches and techniques can be 
implemented to enhance the process of capturing an accurate electrical representation of the 
5 responses uttered by the student. For example, the microphone 1802 can be implemented as a 
so-called horn microphone having a conically-shaped input element designed to funnel the 
speech input into the transducer which capmres the speech information. Additionally, or in 
the alternative, echo cancelling techniques and other noise reduction signal processing 
techniques, which signal processing techniques are well known in other arts, e.g., cellular 

10 telephones, can also be added to improve the captured speech information. Once captured, the 
student's responses are recorded e.g., in any of the memory devices associated with the 
student's workstation 10 (e.g., hard drive 22), the LAN 44 or the teacher's workstation. 

Having captured the smdenfs response(s), the next step 1706 in interactive speech 
education techniques according to the present invention is to evaluate those responses. This 

15 can be accomplished using any suitable software speech engine 1808 designed for the puqjoses 
of speech evaluation. However, Applicants have determined that it is preferable that the 
software engine be able to analyze the speech object responses at a phoneme level rather than a 
word level. Accordingly the software engine which is selected for this task should be capable 
of breaking down the captured responses into their individual phonemes and grading the 

20 pronunciation of each phoneme. An example of this type of software engine is that produced 
by Idioma Ltd., an Israeli company having a place of business at 15 Maskit Street, Herzeliya, 
46733, Israel. The interested reader can find additional information about this exemplary 
software engine at http://www.idioma.co.il, the disclosure of which is expressly incorporated 
here by reference. Those skilled in the art will appreciate, however, that other speech 

25 evaluation engines could also be used to implement the present invention. 

The output of the software engine 1806 is further processed by the application software 
1804 running on workstation 10 to determine an appropriate reinforcer based on the evaluated 
responses. For example, the software engine 1806 may provide as an output a numerical 
grade of each phoneme in the recorded version or versions of the speech object. This output 

30 can then be evaluated by the application software to provide a suitable reinforcer and/or to 
determine and perform a repetitive exercise intended to correct a mispronounced word. 
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For example, suppose that the software engine 1806 evaluates the phonemes in two 
captured versions of the word "theatrical" and outputs averaged numerical grades of 90%, 
92%, 84% and 99% for the four phonemes which comprise this speech object. The 
application software 1804 would receive these numerical grades and compare them with a 
5 threshold, which may be settable by the teacher at his or her workstation. In this example, 
suppose that the threshold is 80% such that this was considered by the system to be correctly 
pronounced. The application software 1804 would make this determination and command the 
GUI 1802 to output a positive reinforcer at step 1708, e.g., one of the educational characters 
saying "Good Job Ellen!". 

10 Alternatively, suppose that the first phoneme "the" in the speech object "theatrical" is 

mispronounced (i.e., below a certain threshold of identity as compared with the correct 
enunciation of this phoneme) as determined by application software based on its evaluation of 
the output of software engine 1806, but that the remaining phonemes are pronounced 
correctly. In this case, the application software 1806 can command GUI 1804 to output a 

15 repetitive reinforcer, e.g., by having an educational character repronounce the word 

"theatrical" with emphasis on the first phoneme and/or displaying the word broken down into 
its phonemes with the first phoneme highlighted. 

As mentioned earlier, the decision-making as to the manner and stimuli in which examples and 
remedial examples are output on the part of application software 1806 can also be influenced 

20 by a student's recorded history of exercise-mistake-corrective example-response to enhance the 
leaming process. For example, if this particular student has a recorded history of correcting 
mistakes with fewer iterations using audible emphasis for incorrectly pronounced phonemes, 
then this type of remedial reinforcement can be provided at step 1708. Application software 
1806 can also decide to perform another iteration of the flow illustrated in Figure 17 if the 

25 grades output by speech engine 1808 warrant retrying that particular speech object. 

Although the foregoing description has been directed to an exemplary system for 
implementing educational methods according to the present invention, one skilled in the art 
will appreciate that many other systems or variations of the system described above could 
readily be configured to implement an educational method according to the present invention. 

30 For example, other types of memory storage devices and communication systems linking the 
student workstations to the teacher workstation could be readily implemented. Methods 
according to the present invention are also not limited to a single group of student terminals 
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linked to a single teacher's workstation, but could include an entire school's workstations or 
more. 

Moreover, one skilled in the art will readily appreciate that emerging multimedia 
technologies and computer developments readily lend themselves to integration with various 
5 embodiments of the present invention. For example, expert systems could be used as part of 
the courseware so that the system develops its own questioning and remedial presentation 
techniques. Moreover, the advent of computer voice synthesis and recognition provides 
another alternative for both student input and system audio output. 

The foregoing description which sets forth exemplary preferred embodiments of a 
10 novel educational method and system for carrying out such a method is merely intended to be 
illustrative of the present invention rather than restrictive or limitive thereof. On the other 
hand, the scope of the invention is set forth in the appended claims which are intended to 
encompass what is directly set forth therein as well as reasonable equivalents thereof including 
those expressly and unplicitly set forth in the disclosure. 
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1 . An educational speech teaching method using an interactive audio-visual 
computer system comprising the steps of: 

outputting, by said computer system, at least one example of a speech object; 

prompting, by said computer system, a student to repeat said speech object; 

storing said repeated speech object in said computer system; 

evaluating said stored, repeated speech object using a speech evaluation engine 
to generate grades associated with said stored, repeated speech object on a phoneme level; and 

determining, by said computer system, whether to provide another example of 
said speech object or to output a positive reinforcer based on said grades generated by said 
speech evaluation engine. 

2. The method of claim 1, wherein said step of outputting at least one example of 
a speech object further comprises the step of: 

displaying an educational character that pronounces said speech object and 
simultaneously outputting an audible example of said speech object. 

3. The method of claim 1, wherein said step of outputting at least one example of 
a speech object further comprises the step of: 

displaying a visual representation of said speech object broken down into its 
composite phonemes. 

4. The method of claim 1, wherein said step of outputting further comprises the 

step of: 

repeating the output of said speech object a plurality of times. 

5. The method of claim 1 , wherein said step of storing said repeated speech object 
in said computer system further comprises the step of: 

capturing said repeated speech object by microphone; and 
recording an electrical representation of said speech object in a memory device 
associated with said computer system. 
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6. The method of claim 1 , wherein said step of determining, by said computer 
system, whether to provide another example of said speech object or to output a positive 
reinforcer based on said grades generated by said speech evaluation engine further comprises 
the step of: 

comparing values of said grades generated by said speech evaluation engine 

with a threshold; and 

selectively outputting said positive reinforcer when all of said grades exceed 

said threshold. 

7. The method of claim 1 , wherein said step of determining, by said computer 
system, whether to provide another example of said speech object or to output a positive 
reinforcer based on said grades generated by said speech evaluation engine further comprises 
the step of: 

comparing values of said grades generated by said speech evaluation engine 

with a threshold; and 

selectively outputting a remedial example of said speech object when at least 

one of said grades is less than said threshold. 

8. The method of claim 7, wherein said remedial example is different than said at 
least one example. 

9. The method of claim 8, wherein a style of said remedial example is selected 
based upon an historical performance of said student. 

10. A computer system for implementing a computer-aided, speech teaching 
method comprising: 

means for outputting, by said computer system, at least one example of a speech 

object; 

means for prompting, by said computer system, a student to repeat said speech 

object; 

means for storing said repeated speech object in said computer system; 
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means for evaluating said stored, repeated speech object using a speech 
evaluation engine to generate grades associated with said stored, repeated speech object on a 
phoneme level; and 

means for determining, by said computer system, whether to provide another 
example of said speech object or to output a positive reinforcer based on said grades generated 
by said speech evaluation engine. 

1 1 . The system of claim 10, wherein said means for outputting at least one example 
of a speech object further comprises: 

means for displaying an educational character that pronounces said speech 
object and simultaneously outputting an audible example of said speech object. 

12- The system of claim 10, wherein said means for outputting at least one example 
of a speech object further comprises: 

means for displaying a visual representation of said speech object broken down 
into its composite phonemes. 

13. The system of claim 10, wherein said means for outputting further comprises: 
means for repeating the output of said speech object a plurality of times. 

14. The system claim 10, wherein said means for step of storing said repeated 
speech object in said computer system further comprises: 

a microphone for capturing said repeated speech object; and 

a memory device for recording an electrical representation of said speech 

object. 

15. The system of claim 10, wherein said means for determining, by said computer 
system, whether to provide another example of said speech object or to output a positive 
reinforcer based on said grades generated by said speech evaluation engine further comprises: 

means for comparing values of said grades generated by said speech evaluation 
engine with a threshold; and 
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raeans for selectively outputting said positive reinforcer when all of said grades 
exceed said threshold. 

16. The system of claim 10, wherein said means for determining, by said computer 
system, whether to provide another example of said speech object or to output a positive 
reinforcer based on said grades generated by said speech evaluation engine further comprises: 

means for comparing values of said grades generated by said speech evaluation 

engine with a threshold; and 

means for selectively outputting a remedial example of said speech object when 

at least one of said grades is less than said threshold. 

17. The system of claim 16, wherein said remedial example is different than said at 
least one example. 

18. The system of claim 17, wherein a style of said remedial example is selected 
based upon an historical performance of said student. 



19. An educational speech teaching method using an interactive audio-visual ^ 
computer system comprising the steps of: 

(a) outputting, by said computer system, at least one example of a speech object; 

(b) repeating, by a student, said speech object; 

(c) storing said repeated speech object in said computer system; 

(d) evaluating said stored, repeated speech object to generate grades associated with 
said stored, repeated speech object; and 

(e) determining, by said computer system, whether to repeat steps (a)-(d) for said 
speech object or to output a positive reinforcer based on said grades generated by said 
computer system. 

20. The method of claim 19, wherein if steps (a)-(d) are repeated, then a remedial 
example is output in step (a), which remedial example is different than said at least one 
example. 
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